Image processing device and elevator mounting it thereon

ABSTRACT

An image processing apparatus detecting a wheelchair passenger for an elevator. This image processing apparatus includes a TV camera installed so that a hail where passengers are waiting is monitored through the camera, which is located in the vicinity of the elevator door, a face area extraction unit, a wheelchair area estimation unit, and a wheelchair existence judgment unit. The image processing apparatus is operated in the following manner, one or a several passengers&#39; face area is identified by the face area extraction unit in the image picked-up by the TV camera, and the position of the face area in the image is detected. If the passenger is sitting in a wheelchair, the area where the wheelchair is likely to be in the image is estimated by the wheelchair area estimation unit from the position of the face area. The estimated area where the wheelchair is likely to be is checked by the wheelchair existence judgment unit, to determine whether the wheelchair is present.

TECHNICAL FIELD

This invention relates to an image processing apparatus which judges thecondition of passengers of an elevator or the like, and an elevatorprovided with this image processing apparatus.

BACKGROUND ART

In the operation of the elevator, it is important to discriminatewhether a passenger in an elevator car is a physically handicappedperson using a wheelchair or an able-bodied person. The reason is thatit becomes possible to provide safe use of the elevator to a passengerin a wheelchair, by changing an operation mode, such that when anordinary able-bodied person is using the elevator, the operation mode isin a normal operation mode, and when a passenger in a wheelchair isusing the elevator, it is changed to a wheelchair mode.

As the wheelchair mode, for example, the elevating speed of the car ismade lower than the normal speed, and at the time of stopping on eachfloor, the landing accuracy of the car with respect to the hall floor isadjusted more precisely than the normal case. When the able-bodied isusing the elevator, the elevating speed of the car is not made low, andthe landing accuracy is not particularly increased, thereby the transittime of the elevator is reduced, and hence the operation efficiency canbe improved.

It is necessary to automatically recognize whether a passenger in awheelchair is trying to use the elevator, in order to automaticallychange the operation mode of the elevator depending on the kinds of thepassenger.

Conventionally, no image processing device has been invented ordeveloped for this purpose, and it is necessary for the passenger in awheelchair to operate a push button dedicated to the wheelchairpassenger by himself/herself, when he/she calls the car, or to operatethe similar wheelchair button in the car after he/she enters into thecar, to thereby give an instruction to change the operation mode withrespect to the elevator system.

With regard to image recognition of the elevator passenger, for example,a measuring device of the number of waiting passengers is disclosed inJapanese Patent Application Laid-Open No. 6-92563. FIG. 16 and FIG. 17are diagrams which explain the contents of the technique described inthis publication.

FIG. 16 shows the configuration of a conventional number measuringdevice. In this figure, the device comprises a camera 1001, an imageprocessor 1002, a neutral network 1003, a number judgment unit 1004, atransmission unit 1005, and an elevator control unit 1006. The TV camerais installed on the ceiling located above the waiting passengers' heads,and is directed vertically downwards.

At first, an image is picked up by the TV camera. The input image 1010in FIG. 17 shows that three people can be seen. An image picked-up inthe situation that there is no person, that is, a background image 1011is prepared beforehand, and the input image is subjected to adifferential processing, thereby a human area appears as a difference.An example of the differential image is shown in 1012 of FIG. 17. Thisis binarized to thereby clearly show the human area. The result ofbinarization is shown in 1013 of FIG. 17. The series of processing isexecuted by the image processor 1002.

Then, the binarized image is divided into twelve blocks, and the dividedimages are input to the neutral network 1003. The input data isprocessed by the neutral network which has performed learningbeforehand, and an output signal is processed by the number judgmentunit 1004, to thereby judge the number of passengers. This result istransmitted to the elevator control unit 1006 by the transmission unit1005.

In the image processing device disclosed in the above publication, if aplurality of passengers is seen in an overlapped state, the image areacorresponding to an individual passenger is not clearly separated, andin the binarized image, such a phenomenon appears that a plurality ofpassengers corresponds to one area. In the conventional art, the camerais installed overhead facing downwards in order to avoid thisphenomenon. However, since there is such a problem in an elevator hallthat installation of a camera on the ceiling increases the cost, thereis a demand to install the camera in the elevator car or in the outerframe of the elevator door. That is, under a normal camera installationcondition, there is a problem in that image recognition is likely to befailed, since the individual passenger area is not separated in theimage.

Further, a method for extracting a face image to recognize an individualperson is disclosed, for example, in Japanese Patent ApplicationLaid-Open No. 2000-193420. It is described here that each position ofeyes is detected from the feature of the ridge of the nose, but it isnot to judge the condition of the person, that is, whether there is awheelchair with this person.

In order to solve the conventional problems, it is an object of thisinvention to provide an image processing apparatus which judges thecondition of a photographed individual, and more specifically to providean image processing apparatus which can automatically recognize whethera passenger who is getting on the elevator sits on a wheelchair, and anelevator provided with this image processing apparatus.

In the invention of this application, after a face area is extractedfrom a photographed image, a second area assuming, for example, awheelchair is extracted from this area to thereby judge the existence(condition) of a wheelchair.

DISCLOSURE OF THE INVENTION

The image processing apparatus according to the present inventioncomprises a face area extraction unit which extracts a first areaincluding a person's face from a photographed person's image, adetection area estimation unit which estimates a desired detection areapreviously registered within the image using the area extracted by theface area extraction unit, and a judgment unit which judges whether animage in a second area extracted by the detection area estimation unitcoincides with a previously registered condition. By this configuration,individual judgment of persons can be accurately performed based on theface area, and an information area which is desired to be known abouteach of the persons can be quickly extracted based on the information,to thereby obtain the person's information.

The image processing apparatus according to another invention comprisesa TV camera installed so as to pick up an image of the outside of acompartment from the vicinity of a door which separates the inside andoutside of the compartment, a face area extraction unit which detectsthe position of a person's face area from the image picked-up by the TVcamera, a wheelchair area estimation unit which estimates an areacorresponding to a wheel chair portion from the image using theextracted face area position when the person is assumed to be on awheelchair, and a wheelchair existence judgment unit which judgeswhether a wheelchair exists from the image information for the estimatedwheelchair area. By this configuration, the position of a passenger isestimated based on its face area, and the position at which thewheelchair should be located is estimated based on this position toperform image processing. Hence, the image processing area is defined,and whether a wheelchair exists is judged quickly and reliably.

The image processing apparatus further comprises a distance measurementunit which measures a distance up to the face area extracted by the facearea extraction unit, and the wheelchair area estimation unit isconfigured so as to estimate a wheelchair area using the measureddistance data, and hence the accuracy of judgment of existence of awheelchair is improved. That is, at the time of detecting the positionof a passenger's face, the height of the passenger's face portion fromthe floor is calculated by estimating the position of the passenger'sface in the three-dimensional space. If the height is beyond a presetrange, it can be clearly judged that the passenger is not sitting on awheelchair. If it is within the range, an area where the wheelchair islikely to exist in the image is estimated when it is assumed that thepassenger sits on a wheelchair, and the estimated area where thewheelchair exists is checked by the wheelchair existence judgment unit,to thereby judge if the wheelchair exists. Therefore, judgment accuracyis improved.

The distance measurement unit comprises an optical beam generation unitwhich generates a spot-like or slit-like optical beam, an optical beamscanning unit which scans so that the generated optical beam isprojected to an area to be measured, and a reflected light detectionunit which detects a reflected light of the scanned and projectedoptical beam. The distance measurement unit measures a distance up to anobject based on a scanning angle of the optical beam and the positionwhere the reflected light is detected. Hence, distance measurementbecomes easily possible.

Further, the image processing apparatus comprises a range gate settingunit which sets a range threshold in order to judge that a personapproaches a certain range from the compartment, and an image pickuptiming determination unit which determines a timing at which an image ispicked up through recognition by the distance measurement unit that aperson comes into the range gate, in which an image is picked up by theTV camera according to the determined timing. Hence, the image iscaptured efficiently and reliably. Further, since an image is picked upat a point of time at which a range between a passenger approaching thecamera and the camera is substantially constant, the passenger's faceand the wheelchair can be picked up on the image in a substantiallyuniform size. Hence, the reliability of image recognition can beimproved.

The elevator according to the present invention is an elevator providedwith the image processing apparatus according to any one of the aboveinventions, a TV camera is the TV camera which picks up an image of apassenger from the vicinity of the elevator door, and informationobtained by the wheelchair existence judgment unit is input into acontrol unit of the elevator. Hence, it becomes possible to change theoperation condition of the elevator corresponding to the passenger,thereby a high-performance elevator capable of providing morecomfortable operation can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 and FIG. 2 are block diagrams which explain the configuration ofan image processing apparatus according to a first embodiment of thepresent invention, and FIG. 3 is a flowchart which shows the operationflow of the image processing apparatus.

FIG. 4 is a diagram which explains an image processing apparatusaccording to a second embodiment of the present invention, and whichshows the situation that three passengers are going to get on anelevator from outside. FIG. 5 is a schematic diagram which explains asubject image to be processed by the image processing apparatusaccording to the second embodiment, which is obtained by photographingthe passengers in FIG. 4. FIG. 6 is a block diagram which explains theconfiguration of the image processing apparatus according to the secondembodiment, and FIG. 7 is a flowchart which shows the operation flow ofthe image processing apparatus.

FIG. 8 is a block diagram which explains the configuration of an imageprocessing apparatus according to a third embodiment, and FIG. 9 is aflowchart which shows the operation flow of the image processingapparatus.

FIG. 10 is a block diagram which explains the configuration of an imageprocessing apparatus according to a fourth embodiment, and FIG. 11 is aflowchart which shows the operation flow of the image processingapparatus.

FIG. 12 is a block diagram which explains the configuration of an imageprocessing apparatus according to a fifth embodiment, and FIG. 13 is aflowchart which shows the operation flow of the image processingapparatus.

FIG. 14 is a block diagram which explains the configuration of an imageprocessing apparatus according to an other embodiment of the presentinvention, and FIG. 15 is a flowchart which shows the operation flow ofthe image processing apparatus.

FIG. 16 and FIG. 17 are respectively a block diagram which explains theconfiguration of a number measuring device related to image recognitionof elevator passengers in the conventional art, and a diagram whichexplains the processing procedure.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of this invention will now be explained with reference tothe drawings.

First Embodiment

FIG. 1 and FIG. 2 are block diagrams which explain the configuration ofan image processing apparatus according to a first embodiment of thepresent invention, and FIG. 3 is a flowchart which shows the operationflow of the image processing apparatus.

In these figures, reference numeral 1 denotes an image pickup devicesuch as a TV camera, 2 denotes a face area extraction unit whichextracts a face area 2 b from a photographed picture 2 a. 3 denotes adetection area estimation unit which estimates a detection area 3 cregistered beforehand, based on a photographed face area 3 b(corresponding to 2 b) portion of a photographed picture 3 a(corresponding to 2 a), 4 denotes a unit which judges by comparisonbetween the image information of 3 c and the preliminarily registeredcondition, and 5 denotes an output of the judgment result.

The image processing procedure will be explained below based on theflowchart in FIG. 3.

At first, in step 101 (hereinafter, described as ST101 in the figure), agrayscale picture is input from the TV camera. This picked-up picture isstored in a memory. In step 102, the face area extraction unit 2extracts a human face area 2 b in the picture 2 a. As the human facearea extraction algorithm, for example, an approach based on extractionof a flesh-colored portion in a color picture is used. When a pluralityof persons exists in the visual field, a plurality of areas areextracted and output. The face area extraction unit 2 then continuouslymeasures the position of the face area in step 103. In this embodiment,for example, the position of a representative point in the area isexpressed by values of barycenter of the picture in the area.

In step 104, the detection area estimation unit 3 performs areaestimation of a desired detection area based on the position of theextracted face area. In FIG. 1, a wheelchair area is extracted. Anexample of the estimation approach of the wheelchair area will beexplained in detail. The width of the face area extracted by the facearea extraction unit is measured. By comparing this width with thereference data related to the width of an ordinary human face, thedistance of the face area from the camera can be estimated. It ispossible to get to know a depression angle of the installed camera, thatis, an angle between the optical axis of the camera and the horizontalline by measuring this depression angle at the time of installing thecamera, and therefore the height of the face area from the floor isestimated using the distance of the face area from the camera and thedepression angle of the camera. Further, it is possible to assume theposition of the wheelchair to be located below by a certain distancewith respect to the center of the passenger's face. The area where thewheelchair can exist in the picture can be set using these data.

In step 105, the estimated wheelchair area is searched, to judge ifthere is a wheelchair. The detection of the wheelchair is performed byusing geometrical features of the wheelchair, such as the seat on whichthe passenger sits, and two wheels existing on the opposite sidesthereof, and by color information and the like. The information can beused for comparison and judgment as preliminarily registered data.

When it is judged that a wheelchair exists within the area by thewheelchair judgment unit, it is judged that the passenger is sitting onthe wheelchair, to thereby generate a wheelchair detection signal, andthe series of processing is terminated.

The example explained above is provided to judge the existence of thewheelchair shown in FIG. 1, but this example is employed in the samemanner as explained above for the judgment of existence of a hat on thehead as shown in FIG. 2, for the purpose of security. In FIG. 2, theupper part of the face area 2 b (3 b) is further analyzed, and bycomparing this part with the estimation of an ordinary head shapewithout wearing the hat, and also by the color information, it can bejudged whether the passenger puts on a hat.

In the above embodiment, the example in which only one face area isextracted in the visual field of the TV camera has been explained.However, if there is a plurality of face areas, by repeating the stepsof step 102 to step 105 for each face area in the same manner, judgmentbecomes possible.

In this embodiment, a barycentric position of the area is used as theface area position, but the center of a circumscribed quadrilateral ofthe face area may be used, or a characteristic portion in the face area,such as eyes or nose may be detected by detailed image processing tothereby calculate the representative position.

In this embodiment, the approach based on extraction of theflesh-colored area using a color picture has been explained for the facearea extraction unit, but it is a matter of course that the same effectas that of this embodiment can be obtained by extracting theflesh-colored area by a pattern matching technique in which a monochromepicture is input to detect the face pattern including face parts such aseyes and nose.

Further, in this embodiment, as the wheelchair judgment unit, theapproach which judges the existence of a wheelchair from its geometricalfeature has been explained as one example. However, this embodiment canbe executed also by performing judgment using other features within thewheelchair area. The features, such as the situation of the legs of thepassenger who is sitting on the wheelchair, can help discrimination of awheelchair passenger from pedestrian able-bodied people.

As explained in this embodiment, the person's face area is firstextracted from an image including the person, a desired detection areais estimated and detected from the information of the face area, andthen it is judged whether a wheelchair exists or the person puts on ahat. Hence, the processing speed is improved and the final judgmentaccuracy is also improved by sequentially cutting out only the necessaryinformation from the image.

Second Embodiment

A second embodiment of the present invention will now be explained. FIG.4 explains an image processing apparatus in one embodiment of thepresent invention, and schematically shows the situation that passengersare going to get on an elevator from outside when viewed from the side.FIG. 5 is a schematic diagram which explains a subject image to beprocessed by the image processing apparatus of this embodiment, which isobtained by photographing the state shown in FIG. 4 by a camera. FIG. 6is a block diagram which explains the configuration of the imageprocessing apparatus according to this embodiment, and FIG. 7 is aflowchart which shows the operation flow of the image processingapparatus.

In these figures, reference numeral 11 denotes a TV camera installed inthe upper part of an external door of the elevator 16, which takes apicture of passengers 17 a and 17 b who are going to get on theelevator. For example, passengers are assumed to include the passenger17 a using a wheelchair and the passenger 17 b who does not use awheelchair. 12 denotes a face area extraction unit, 13 denotes awheelchair area estimation unit, 14 denotes a unit which judges theexistence of a wheelchair, and 15 denotes a wheelchair existencedetection signal.

The operation of this image processing apparatus will be explained belowbased on the flowchart in FIG. 7. In step 201, a grayscale picturephotographed by the TV camera 11 is input. The TV camera is installed onthe upper part of the external frame of the external door, and the sightline thereof is directed to a direction so that the camera can catchwhole passengers waiting for the elevator. The picked-up image is storedin an image memory. This image, that is, the picture photographed by theTV camera 11 in FIG. 4 is shown, for example, in FIG. 5.

In step 202, a human face area is extracted from the image by the facearea extraction unit 12. As the human face area extraction algorithm,for example, an approach based on extraction of a flesh-colored portionin a color picture is used. When a plurality of people exists in thevisual field, a plurality of areas are extracted and output. Thiscorresponds to each hatched area in the respective areas indicated bydotted lines. Then, in step 203, the positions of the face areas arecontinuously measured. In this embodiment, the position of arepresentative point in each area is expressed by values of barycenterof the picture in the area, as in the first embodiment.

In step 204, the position of a wheelchair, at which the wheelchair issupposed to exist if the person is in a wheelchair, is estimated basedon the extracted face area position using the wheelchair area estimationunit 13. As described in the first embodiment, this is performed by amethod of measuring the width of the face area extracted by the facearea extraction unit, or the like. Thereby, the distance of the facearea from the camera can be estimated by comparing the distance with thereference data related to the width of an ordinary human face. It ispossible to get to know a depression angle of the installed camera, thatis, an angle between the optical axis of the camera and the horizontalline by measuring the angle at the time of installing the camera, andtherefore the height of the face area from the floor is estimated usingthe distance of the face area from the camera and the depression angleof the camera. Further, it is possible to assume the position of thewheelchair to be located below by a certain distance with respect to thecenter of the passenger's face. The area in the picture in which thewheelchair possibly exists can be set using these data.

In step 205, the wheelchair existence judgment unit 14 searches theestimated wheelchair area to thereby judge if there is a wheelchair. Thedetection of the wheelchair is performed by using geometrical featuresof the wheelchair, such as the seat on which the passenger sits and twowheels existing on the opposite sides thereof, and by using colorinformation and the like.

Particularly, the details of judgment of existence of a wheelchair usingthe grayscale picture will be explained next.

In order to judge the existence of a wheelchair, a learning pattern iscreated beforehand, and a lot of learning data is prepared. The learningdata is image data including a wheelchair photographed in the picture,and a lot of data is prepared by changing passengers' clothes, types ofthe wheelchair, time of photographing or the like in various ways. Aplurality of images of the area in the vicinity of the legs ofpassengers in wheelchairs are cut out from these learning images, andare rearranged to one-dimensional signals in accordance with a certainrule (for example, raster scanning), and plotted on a multi-dimensionalspace, wherein the luminance of each pixel is designated as a value ofeach axis. The areas consisting of plot point groups corresponding tothe plurality of these images including the wheelchair are designated aslearning point groups including a wheelchair.

The procedure of judging the existence of a wheelchair by using thislearning pattern will now be explained. An area in which a wheelchairwill exist is extracted based on the face image area information. In thesame manner as for the learning data creation, the extracted image areais plotted in the same multi-dimensional space. This is an unknown datapoint. Then, the distance between the unknown data point and each pointconstituting the plurality of learning point groups is respectivelycalculated, and is compared with a preset threshold. If there are theunknown data and a learning data point existing within the range belowthe threshold, this unknown data is judged to include a wheelchair.

In step 206, if it is judged that a wheelchair exists in the area by thewheelchair judgment unit, it is judged that the passenger is sitting onthe wheelchair, and a wheelchair detection signal is generated, therebythe series of processing is terminated.

In this embodiment, an example has been explained in which only one facearea is extracted in the visual field of the TV camera. However, ifthere is a plurality of face areas, by similarly repeating the steps ofstep 202 to step 206 for each face area, it can be recognized if each ofthe plurality of passengers approaching the elevator car is sitting onthe wheelchair.

As explained in the first embodiment, the center of a circumscribedquadrilateral of the face area may be used as the face area position, ora characteristic portion in the face area such as eyes or nose may bedetected through further specific image processing to thereby calculatethe representative position.

Further, as explained in the first embodiment, it is a matter of coursethat the similar effect to that of this embodiment can be obtained byusing the pattern matching technique, for the face area extraction unit,in which a monochrome picture is input to detect the face patternincluding face parts such as eyes and nose.

As explained in the first embodiment, as the wheelchair judgment unit,the approach of judging the existence of a wheelchair from a geometricalfeature has been explained as one example. However, this embodiment canbe executed also by performing judgment using other features within thewheelchair area, which can discriminate the wheelchair passenger frompedestrian able-bodied people, such as the situation of the legs of apassenger who is sitting on the wheelchair.

As shown in this embodiment, in the process during which the elevatorcar arrives and is landed and passengers waiting for the car in theelevator hall get on the car, the image of the passenger can be obtainedone by one in the state without being concealed by other people. Hence,stable image recognition becomes possible. Further, in the recognition,passenger's position is estimated based on the face area, which is thearea at a high position unlikely to be concealed by other people, andthe position of a wheelchair is also estimated, to thereby define theimage processing area. Thereby, there is the effect that the wheelchairexistence judgment can be stably performed.

Third Embodiment

A third embodiment of the present invention will now be explained. Thisembodiment has such a configuration that a distance measurement unitwhich measures the distance up to a subject image is added to that ofthe second embodiment in addition to the camera which picks up the imageprovided therein.

FIG. 8 is a block diagram which explains the configuration of an imageprocessing apparatus according to this embodiment, and FIG. 9 is aflowchart which shows the operation flow of the image processingapparatus.

In the figures, reference numerals 11 to 15 are the same as those in thesecond embodiment. Reference numeral 18 denotes the distance measurementunit. Though not shown, the distance measurement unit in this embodimentis an example based on a method for analyzing a stereo image using atwin-lens TV camera.

The operation procedure will be explained with reference to FIG. 9.Since steps 301 to 303 are the same as those in the first embodiment,the steps will be explained briefly. At first, the face area isextracted in step 302 based on the image input by the TV camera in step1, and the position of the face area is detected using a barycentricmethod or the like in step 303. Since the TV camera is a twin-lensreflex camera, the processing in step 301 to step 303 is carried outrespectively based on the images of the respective cameras.

In step 304, distance measurement is executed by the distancemeasurement unit 18. In this embodiment, a twin-lens stereo method isemployed as a distance measurement method. That is, as the TV camera,two TV cameras whose geometrical position is known are disposed, and byobtaining the correspondence between respective pixels of the imagecaptured by each camera, a distance distribution of the object ismeasured using the principle of triangulation. The distance distributionis stored as a distance image in which the distance from the camera tothe object is stored in each pixel.

In step 305, the height of the face area from the floor is calculated bythe wheelchair area estimation unit 13 from the position of theextracted face area in the image and the distance measurement resultcorresponding to the face area. In this embodiment, a positionalrelation between the grayscale picture obtained by the TV camera and thedistance image is previously made through the calibration operation.Since the depression angle of the TV camera is known as in the secondembodiment, the three-dimensional position of the face area isidentified by combining this depression angle with the distance datafrom the camera to the face area. As a result, the height of the facearea from the floor is calculated.

In step 406, the wheelchair area estimation unit 13 compares themeasured height of the face area from the floor with the presetthreshold. In this embodiment, this threshold is set as an average valuebetween the height of the face when ordinary adults are standing and theheight of the face when the adults are sitting on wheelchairs. If theheight of the face area from the floor is larger than the threshold, itis judged that the face area is of a standing passenger, and thereforethe wheelchair detection processing is not performed to end theprocessing. However, if the height of the face area from the floor issmaller than the threshold, it is judged that the face area is quitepossible to be of a wheelchair passenger, and therefore control proceedsto step 307, where the position on the image at which the wheelchair mayexist is estimated.

In step 308 and step 309, the wheelchair existence judgment unit 14judges whether a wheelchair exists by performing detailed investigationwith respect to the estimated wheelchair area, and when it is judgedthat a wheelchair exists, the wheelchair existence judgment unit 14generates a wheelchair detection signal. These operations are the sameas those of step 205 and step 206 in the second embodiment.

In this embodiment, the twin-lens stereo method is used for the distancemeasurement unit 18. However, the similar effect can be obtained byusing other distance measuring approaches having a function of measuringthe distance of the extracted face area from the camera, such as adistance measuring approach using a camera having three or more lenses,or a distance measuring approach in which the distance is measured basedon a blurred picture by a single-lens camera.

In this embodiment, there is shown an example of once forming a datastructure in which a distance image, i.e., each pixel includes distanceinformation from the camera. However, the distance data needs only toinclude the information corresponding to a representative position ofthe extracted face area, and therefore the similar effect can beobtained, for example, by dividing the visual field of the TV camerainto a plurality of blocks and installing a plurality of distancemeasuring sensors of a spot light irradiation type which measures thedistance of the face area from the camera with respect to the centralportion of each block, or by providing a plurality of ultrasonic sensorshaving a narrow directivity.

As shown in this embodiment, accurate estimation of the wheelchair areacan be performed by accurately judging the three-dimensional position ofa passenger, and as a result, the reliability in judging that thepassenger is a wheelchair passenger can be improved.

Fourth Embodiment

A fourth embodiment of the present invention will now be explained.

FIG. 10 is a block diagram which explains the configuration of an imageprocessing apparatus according to this embodiment, and FIG. 11 is aflowchart which shows the operation flow of the image processingapparatus. This fourth embodiment is related to an image processingapparatus which uses a laser ranging sensor as the distance measurementunit in the third embodiment.

In the figures, reference numerals 11 to 15 are the same as those in thesecond and third embodiments. Reference numeral 19 denotes a distancemeasurement control unit, 20 denotes an optical beam generation unit, 21denotes an optical beam scanning unit and 22 denotes a reflected lightdetection unit.

The operation will be explained with reference to the flowchart in FIG.11. FIG. 11 shows another embodiment related to the step 304 in FIG. 9.Hence, START in FIG. 11 indicates the step following step 303 in FIG. 9,and END in FIG. 11 will proceed to step 305 in FIG. 9. An image is firstinput by the TV camera 11, and the face area extraction unit 12 extractsa human face area from the image and measures the position of arepresentative point in the face area.

Subsequently, the distance measurement after step 401 and on isperformed. The distance measurement control unit 19 is a unit whichcontrols the optical beam generation unit 20, the optical beam scanningunit 21, and the reflected light detection unit 22. The principle ofdistance measurement shown in this embodiment is to emit a slit-typelaser beam, pick up the image of the beam which hits an object andreturns by a camera, and measure the distance based on the principle oftriangulation, and it is referred to as a light-section method. Theoptical beam generation unit 20 consists of a semiconductor laser lightemission element and a lens, in which excited and output laser beams areonce converged and the converged beam is further deformed into a slit bya cylindrical lens to generate the slit-like beam. In the first step 401for the distance measurement, the emission pattern of the laser beam isinitialized by the distance measurement control unit 19. The emissionpattern indicates a pattern in which a light is turned on or offsynchronously to the scanning unit, when laser slit beams are scanned.In this embodiment, such a light emission pattern that a light is turnedon momentarily and is turned off in other periods in scanning for onescreen is initialized so that a plurality of patterns are generated inwhich the lighting timing is variously changed.

In step 402, the optical beam scanning unit 21 initializes the controlpattern for the laser beam scanning. This intends that the scanningspeed and range are initialized so that the optical beam scanning unitscans in synchronization with image capturing timing.

In step 403, image accumulation is started by a CCD camera which is thereflected light detection unit 22. In step 404, a galvano-mirror whichis the optical beam scanning unit 21 starts the operation according tothe initialized pattern, and in step 405, the optical beam generationunit 20 controls ON/OFF of the beam according to the initializedpattern. In step 406, it is judged whether the laser beam scanning hasbeen finished. If it has not been finished, the operations in steps 404to 405 are repeated, and if it has been finished, the image accumulationby the reflected light detection unit 22 is terminated in the next step407.

In step 408, the accumulated image of the laser beam reflected light isread out and a position of the image is measured through imageprocessing. The three-dimensional position, i.e., the distance at theposition on the object to which the optical beam has been projected ismeasured, based on the principle of triangulation, from the position ofthe reflected light of the beam on the detected image and the scanningangle of the optical beam scanning unit 21 at the time when the laserbeam is turned ON. Then in step 409, the measured distance is stored inthe memory of the distance measurement control unit 19 as the distancedata.

In this embodiment, the light emission pattern is set so that theslit-like laser beam scanned during the image accumulation period forone image of the CCD camera that is the reflected light detection unit,is turned ON only once. Hence, a distance distribution for one line inthe longitudinal direction in the image accumulation for one image canbe measured. Therefore, by repeating operations in steps 403 to 409while changing the time when the beam is turned ON, laser beams can beirradiated to various positions, and the distance at the irradiatedposition can be measured. As a result, in step 409, the distance imagecan be obtained. The steps for distance measurement are now completed.

The operation in this embodiment is followed by operations as shown insteps 305 to 309 in FIG. 9. That is, the wheelchair area estimation unit13 calculates the height of the face area from the floor from theposition in the image of the extracted face area and from the result ofdistance measurement result corresponding to the face area. Similarly,the wheelchair area estimation unit 13 compares the height of themeasured face area from the floor with the preset threshold to performrough judgment related to the existence of a wheelchair. If the heightof the face area from the floor is smaller than the threshold, it isjudged that the face area is quite possible to be of a wheelchairpassenger, and the position in the image where the wheelchair may existis estimated. In the end, the wheelchair existence judgment unit 14performs detailed investigation for the estimated wheelchair area, andjudges the existence of a wheelchair and when it is judged that there isa wheelchair, the wheelchair existence judgment unit 14 generates awheelchair detection signal.

In this embodiment, the approach of generating distance imagesrepeatedly while changing light emitting timings has been explainedbased on the light-section method for emitting one laser slit beamduring the accumulation period of one image by the CCD camera that isthe reflected light detection unit, as the distance measurement unitcomposed of the optical beam generation unit, the optical beam scanningunit, and the reflected light detection unit. However, the similareffect can be obtained by using a spatial coding method for projecting aplurality of laser slit beams during the accumulation period of oneimage by the CCD camera.

In this embodiment, the example in which the galvano-mirror is employedas the optical beam scanning unit 21 is explained. However, a polygonmirror of a polygonal pillar type may be used.

As shown in this embodiment, there is the effect that stable distancemeasurement can be performed without being influenced by the existenceor the complexity of a pattern in the clothes that the passenger wears,by using the distance measurement unit which projects the optical beam.

Fifth Embodiment

A fifth embodiment of the present invention will now be explained.

FIG. 12 is a block diagram which explains the configuration of an imageprocessing apparatus according to this embodiment, and FIG. 13 is aflowchart which shows the operation flow of the image processingapparatus.

In the figures, reference numerals 11 to 15 and 18 are the same as thosein the second to fourth embodiments. 23 denotes an image pickup timingdetermination unit, and 24 denotes a range gate setting unit.

This embodiment will be explained based on the flowchart in FIG. 13.

At first, in step 501, a range gate is set by the range gate settingunit 24. The range gate stands for a threshold related to a distancebetween the TV camera 11 and a passenger when the passenger approachesthe elevator car when its door opens. For example, the distance betweenthe camera and the passenger required to pickup an image of thepassenger's face area in a desired size by the TV camera 11 is examinedbeforehand and designated as the threshold.

In step 502, the distance between the camera and the passenger ismeasured by the distance measurement unit 18. As a distance measuringapproach, for example, the approach explained in the third or fourthembodiment is employed.

In step 503, the measured distance between the camera and the passengeris compared with the preset range gate threshold. If the distancebetween the camera and the passenger has not reached the range gate, thedistance between the camera and the approaching passenger is measuredagain in step 502. If the distance between the camera and the passengerhas reached the range gate, the timing when the image is picked up isdetermined by the image pickup timing determination unit 23 in step 504.This timing is determined so that the size of the face area of theimaged person in the picked-up image is close to the preset value.

In step 505, the image of the passenger is picked up by the TV camera11. In step 506, the face area is extracted by the face area extractionunit 12 and the position of a representative point in the face area isdetected in step 507. In step 508, an area where a wheelchair may existis estimated based on the position of the face area extracted by thewheelchair area estimation unit 13. The operation of the wheelchair areaestimation unit is the same as that explained in the second embodiment.

In step 509, the inside of the estimated wheelchair area isimage-processed and the existence of a wheelchair is judged by thewheelchair existence judgment unit 14. As a result, if it is judged thatthere is a wheelchair, a wheelchair detection signal 15 is output instep 510, and the series of operation is finished.

In this embodiment, there has been explained the approach shown in thesecond or third embodiment as the distance measurement unit. However,this unit needs only to be a sensor having a function to find that thepassenger has approached the camera within a certain range, and needlessto say, the similar effect can be exerted by using, for example, aphotoelectronic sensor, an ultrasonic sensor, or a pyroelectric humansensible sensor.

In this embodiment, the approach which is used only for comparison ofthe output of the distance measurement unit with the range gate set inadvance has been explained. However, if any unit which can obtain adistance image is employed as the distance measurement unit, it is alsopossible by a device having the same configuration as that in FIG. 12,to increase the reliability of the wheelchair area estimation bydetermining the distance of the face area extracted by the face areaextraction unit from the cameras, as shown in the embodiment related tothe second invention.

As shown in this embodiment, images can be picked up at a point of timeat which the distance between a passenger approaching the camera and thecamera is substantially constant. Therefore, the passenger's face andthe wheelchair can be picked up substantially in a uniform size on theimage, and hence there is the effect that the reliability of imagerecognition can be improved.

Sixth Embodiment

A sixth embodiment of the present invention will now be explained. Thisembodiment relates to an elevator apparatus provided with the imageprocessing apparatus of the above-described second to fifth embodiments,in which the wheelchair existence judgment signal output in the aboveembodiments is input to the control unit of the elevator to therebycontrol the operation of the elevator.

FIG. 14 is a block diagram which explains the configuration of an imageprocessing apparatus according to this embodiment, and FIG. 15 is aflowchart which shows the operation flow of the image processingapparatus.

In these figures, reference numerals 11 to 15 are the same as those inthe above embodiments, and 25 denotes an elevator control unit.

The wheelchair recognition signal 15 obtained by the image processing inthe second to fifth embodiments is used and input to the elevatorcontrol unit as follows. The control operation includes the followings.

-   -   (a) opening/closing speed of the elevator door (the door is        slowly opened for a wheelchair passenger),    -   (b) opening time of the elevator door (when the passenger uses a        wheelchair, the door is controlled so as to be opened for a        longer period),    -   (c) adjustment of the elevator floor (with regard to a control        of a difference in step between the elevator car and the outside        floor, the car is controlled so as to agree with the floor level        more accurately when a wheelchair passenger gets on the        elevator, to thereby eliminate the difference in level),    -   (d) guidance by voice (voice announcement such as “The elevator        is now operated in a barrier-free mode”),    -   (e) elevator moving speed (when a wheelchair passenger is on,        the maximum movement speed and acceleration of the elevator are        decreased),    -   (f) push button control inside the elevator (when a wheelchair        passenger is on, a wheelchair dedicated button installed at a        lower position in the car is made effective, and at other times,        the button is made invalid, or it is made effective only when        the button is pressed in a special manner, such as being pressed        for a long time), and    -   (g) control of a special sensor inside the elevator car (when a        wheelchair passenger is on, a sensor dedicated to a physically        handicapped person, such as voice recognition, installed in the        car is made effective) By performing these controls, the        operation of the elevator which gives priority to a wheelchair        passenger can be performed, and it is thereby possible to        increase the operation efficiency and provide comfortable        operation conditions to passengers.

FIG. 14 of this embodiment mainly shows the configuration of the imageprocessing apparatus explained in the second embodiment, but it is amatter of course that it may be the configuration of any imageprocessing apparatus including the distance measurement unit shown inthe third embodiment, the distance measurement control unit, the opticalbeam generation unit, the optical beam scanning unit and the reflectedlight detection unit shown in the fourth embodiment, or the distancemeasurement unit, the range gate setting unit, and the image pickuptiming determination unit shown in the fifth embodiment.

INDUSTRIAL APPLICABILITY

The image processing apparatus according to this invention is used toautomatically detect that a passenger using a wheelchair is included inpassengers who are going to get on the elevator, or the like, and alsoused for control, such as changing the operation mode of the elevator inresponse to the detection.

1. An image processing apparatus comprising: an image pick up devicewhich picks up an image of a person; a face area extraction unit whichextracts a first area including a person's face from an image picked upby the image pick up device; a wheelchair area estimation unit in whicha desired detection area is previously registered and which estimatesthe desired detection area using the first area extracted by the facearea extraction unit to obtain an image in a second area; and awheelchair existence judgment unit which judges whether the image in thesecond area extracted by the detection area estimation unit coincideswith a condition of the area previously registered.
 2. An imageprocessing apparatus comprising: a TV camera installed to pick up animage of an outside of a compartment proximate a door which separates aninside of the compartment from the outside of the compartment; a facearea extraction unit which detects a position of a person's face area inthe image picked-up by the TV camera; a wheelchair area estimation unitwhich estimates a wheelchair area corresponding to an approximateposition of a wheelchair and which uses the face area position detectedwhen the person is assumed to be in a wheelchair; and a wheelchairexistence judgment unit which judges whether a wheelchair exists in theimage from the wheelchair area estimated.
 3. The image processingapparatus according to claim 2, further comprising a distancemeasurement unit which measures a distance to the face area extracted bythe face area extraction unit, wherein the wheelchair area estimationunit estimates the wheelchair area using the measured distance data. 4.The image processing apparatus according to claim 3, wherein thedistance measurement unit comprises: an optical beam generation unitwhich generates a spot-like or slit-like optical beam; an optical beamscanning unit which scans so that the optical beam generated isprojected to an area to be measured; and a reflected light detectionunit which detects reflected light of the optical beam, the distancemeasurement unit measuring a distance to an object based on a scanningangle of the optical beam and a position where the reflected light isdetected.
 5. The image processing apparatus according to claim 3,further comprising: a range gate setting unit which sets a rangethreshold in order to judge when a person approaches an establishedrange from the compartment; and an image pickup timing determinationunit which determines timing at which an image is picked up, throughrecognition by the distance measurement unit, when the person comes intothe established range, wherein an image is picked up by the TV cameraaccording to the timing determined.
 6. An elevator including imageprocessing apparatus, the image processing apparatus comprising: a TVcamera installed to pick up an image of an outside of an elevator fromproximate a door which separates the outside from an inside of theelevator; a face area extraction unit which detects a position of aperson's face area in the image picked-up by the TV camera; a wheelchairarea estimation unit which estimates a desired area corresponding to anapproximate position of a wheelchair and which uses the face areaposition detected when the person is assumed to be in a wheelchair; anda wheelchair existence judgment unit which judges whether a wheelchairexists in the image from the wheelchair area estimated, wherein the TVcamera photographs a passenger from proximate the door of the elevator,and information obtained by the wheelchair existence judgment unit isinput into a control unit of the elevator.